4 
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TO ALL WHOM IT MAY CONCERN: 

BE IT KNOWN that we, * and *, residing in the State of *, have invented new 
and useful improvements in a 
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CROSS REFERENCE TO RELATED APPLICATIONS 

1 The present application claims priority from US Provisional application 

2 60/1 99,567, filed 4/25/00. 

BACKGROUND OF THE INVENTION 

3 1 . Field of the Invention: 

4 The present invention is in the field of digital communications systems, 

5 and more particularly, communications systems which transport continuous 

6 digital media such as audio and video. 

7 2. Description of the Prior Art: 

8 One of the most important forms of information in digital communications 

9 systems is continuous media, exemplified by digital audio and digital video. An 

10 example application would be transmission of a movie or live information 

11 between a service provider and a user. The traditional method of providing such 

12 a service is the broadcast network, exemplified by broadcast television or cable 

13 television. Models for future information systems use the concept of integrated 

14 services, where voice, video and structured information services such as the 

15 world-wide web (WWW) are delivered over a single logical transport service, the 

16 packet-switched Internet. 

17 To use such a system, the audio and video information must be encoded 

18 in digital form, and packetized. To reduce the use of network resources, the 
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1 information might be first encoded in digital form and then compressed. The 

2 advantage of compression is a significant reduction in data volume, while the 

3 disadvantage is the complexity of the algorithms required for encoding and 

4 decoding the compressed information. For example, directly encoding NTSC 

5 signals from a television system requires about 100 megabits/second of 

6 bandwidth, meaning that a two hour video, at 720 megabytes per minute, would 

7 require about 100 gigabytes of storage. Encoding the signals using the MPEG 2 

8 compression scheme typically reduces the bandwidth required to about 1.5 

9 megabits/second, or 12 megabytes per minute. With such compression an entire 

10 2 hour movie could be stored in about 1.5 gigabytes of storage. MPEG 2 is 

11 designed to require significantly more computation to encode than to decode, as 

12 it is presumed that service providers can afford to perform the expensive 

13 operations once, in order that the inexpensive decoding operations can be 

14 performed many times by receivers. This assumption is clearly true for a stored 

15 video, where the encoding is done once and then decoding is done whenever the 

16 video is viewed. 

17 When a system is constructed to distribute digitized continuous media to 

18 many users, there are a number of attractive opportunities for architectural 

19 techniques which can reduce system load in addition to any gains achieved by 

20 operations performed on the media such as efficient coding. In particular, a 

21 powerful technique is multicast, where the information is not sent to all possible 

22 recipients, but rather those who indicate interest, perhaps by subscription. To the 

23 degree that sharing can be achieved, e.g., the sharing of a viewing service which 
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1 plays the video into the network, significant savings can be realized. For 

2 example, rather than sending multiple copies of the same video stream, each at 

3 1 .5 megabits/second to a significant population of users, the video might be sent 

4 once via multicast to the set. 

5 Multicasts are typically represented as acyclic directed graphs called 

6 trees, where the server lies at the root of the tree, there are a set of intermediate 

7 nodes interconnected by network links which transport information to leaf nodes, 

8 at which are located users. A key feature of the intermediate nodes is their ability 

9 to replicate information to several nodes in the direction of the information flow, 

10 so that eventually the information travels from the server to all interested leaves. 

11 This basic model assumes that the users are all interested in receiving the 

12 same information at the same time, e.g., a "scheduled" time for a broadcast 

13 event, such as a sports event or a concert. When users join the multicast at 

14 some later time, they may lose the first part of the event (which they may be 

15 interested in) as it is not saved (or "buffered"). For viewing of archived material, 

16 such as replays of videos on demand, the points at which users are interested 

17 and start the viewing will vary sufficiently that multiple time-skewed copies of the 

18 digital continuous media may be being served to users at any given time. 

19 The difficulties with this situation are two-fold. First, the server is busy 

20 sending and resending multiple multicast streams of essentially the same 

21 information. Second, the advantages of multicast accrue to the degree that 

22 information is replicated - unicast, or point-to-point transmission, can be consider 
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1 a degenerate case of multicast, and in fact will be the case when there is no 

2 shared interest in a continuous media stream. Thus, to the degree in which we 

3 can aggregate demand for continuous media streams, we can optimize overall 

4 system performance. 
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SUMMARY OF THE INVENTION 

1 In accordance with the present invention, a communications method utilizes 

2 memory areas to buffer portions of the media streams. These buffer areas are 

3 shared by user applications, with the desirable consequence of reducing 

4 workload for the server system distributing media to the user (client) applications. 

5 The preferred method allows optimal balancing of buffering delays and server 

6 loads, as well as optimal choice of buffer contents for the shared memory buffers. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

1 The novel features believed characteristic of the invention are set forth in the 

2 appended claims. The invention itself however, as well as a preferred mode of use, 

3 further objects and advantages thereof, will best be understood by reference to the 

4 following detailed description of an illustrative embodiment when read in 

5 conjunction with the accompanying drawings, wherein: 



6 Figure 1 is a block diagram illustrating multicast over a network; 

7 Figure 2 illustrates shared use of a buffer; 

8 Figure 3 is a diagram illustrating use of a double buffer; and 

9 Figure 4 is a timing diagram illustrating buffer sharing. 
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DESCRIPTION OF THE PREFERRED EMBODIMENT 

1 It will be appreciated by those skilled in the art that the following 

2 description illustrates techniques that can be utilized in various types of systems. 

3 The preferred embodiment is described in the context of an internet system such 

4 as is generally known in the art. 

5 MULTICAST TRAFFIC 

6 In a store-and-forward packet-switched network, multicast is effected by 

7 multicasting the packets of data as they arrive at a node in the multicast tree. In 

8 practice, the node is a switch, router or other device which receives a multicast 

9 packet on an input port, and sends the packet on one or more output ports in 

10 accordance with the topology of the graph describing the multicast of the data to 

1 1 users. A multicast from a server to a number of users is illustrated in Figure 1 . 

12 To perform this operation, the packet must be stored in a buffer memory. 

13 In practice, the buffer memory may contain a queue of multiple packets. In the 

14 continuous media case, a queue containing a sequence of packets represents a 

15 segment of the continuous media stream. When the queue consists of large 

16 numbers of packets, substantial storage of media can be achieved within the 

17 constraints of the node's memory capacity. 

18 While it is often the case that the buffering is mainly used to effect flow 

19 control, that is, to mitigate rate differences between senders and receivers, the 

20 presence of buffering capability in nodes allows other optimizations where the 
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1 buffer capacity, e.g., the queued packets, can be shared or reused effectively. 

2 The advantage of sharing and reuse is that once the packet has been transmitted 

3 to a buffer, it need not be transmitted from the server to service a user if the user 

4 can be satisfied with the buffer contents rather than with an additional 

5 independent stream sent by the server. 

6 The difficulty in buffer reuse is independent start times for the multicast 

7 streams, as what is in the queue of packets destined to a particular user may not 

8 be the same data as that required by another user. To the degree that work (e.g., 

9 the transmission of data from one node to another) can be avoided through 

10 sharing, the system will perform better. An example of better performance would 

11 be more concurrent users operating with a reduction in bandwidth relative to 

12 similar systems. 

13 SHARED BUFFERS 

14 Each segment of buffered continuous media represents the result of work done 

15 by the network to transport the digitized continuous media to the buffer. Thus, to 

16 the degree that the contents of the buffer can be shared, the work of transporting 

17 the continuous media to the buffer can be shared. An essential observation is 

18 that there is a clear relationship between buffer occupancy and time: each buffer 

19 segment of size B bytes represents a playout time of B/R seconds, where R is 

20 the encoding rate in bytes per second. A related observation was made in John 

21 H. Shaffer, "The Effects of High Speed Netwoks on Wide Area Distributed 

22 Systems", Ph.D. Thesis, CIS, University of Pennsylvania, 1996. 
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1 The central observation used in our approach is that if additional viewers 

2 arrive within B/R seconds of the start time of the original playout, these additional 

3 viewers can utilize the contents of the buffer segment rather than requiring 

4 additional network bandwidth for transmission. To be specific, if the arrival time of 

5 viewers V1 and V2 is separated by less than B/R, they can share a buffer 

6 segment. This is illustrated in Figure 2. 

7 While a related idea was employed in U. Legezda, D. Wetherall and J. 

8 Guttag, "Active Reliable Multicast", Proc. Infocomm 1998, SF, CA to reduce 

9 required bandwidth in a reliable multicast, the sharing was different, in that it was 

10 used for recovery of lost multicast packets rather than as a support mechanism 

1 1 for multicast of digital continuous media. The key basic point of our method is the 

12 use of a per-recipient pointer (an index) into the shared buffer to reflect the 

13 differences in start times. By use of this pointer, the buffer contents can be 

14 effectively shared. 

15 There are a variety of parameters which can be adjusted in the design of 

16 such a system. First, to the degree that whole instances of digital continuous 

17 media (for convenience, we will call these "files") can be buffered, there is 

18 significant advantage to be had in that the buffer management algorithms are 

19 simplified. This is because the algorithms need to spend less effort managing the 

20 differences in buffer refresh rate caused by supporting multiple start times. This 

21 management cost is incurred if the whole file is not available, as to save 

22 bandwidth, the buffer contents must be retained until the last viewer is done with 

23 them, i.e., their pointer has advanced to the end of the buffer. This problem is 
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1 easily addressed if the well-known technique of double-buffering is employed to 

2 manage two buffers of size B and the start time deltas are limited to B/R, as 

3 above. In the double-buffering technique, one buffer is drained by the viewers 

4 while the other is filled from the network, and this solution allows the time- 

5 separated viewers to share the 2*B space. The technique is illustrated in Figure 

6 3. 



7 The same effect can be achieved by limiting the time differences between 

8 V1 andV2toB/(2*R). 

9 OPTIMAL SHARING OF A SINGLE BUFFER 

10 There are a variety of techniques for buffer management which can make 



11 use of the buffer capacity to better store continuous digital media. To review 

12 some of the techniques used in US Patent 5,754,938, issued May 19, 1998 to 

13 Herz et al, and US Application No. 09/024,278, filed Feb 17, 1998, both hereby 

14 incorporated by reference hereinto, the buffer contents can be: 



15 1 . Selected based on statistical modeling of the user based on similarity 

16 measures derived from previous viewing. 

17 2. Can be prefetched in advance of viewing demand in order to smooth 

18 demands on bandwidth. 

19 3. Can be prefetched in anticipation if possible viewing demand based on 

20 similarity measures for the viewers sharing the buffer. 
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1 4. Can be retained in anticipation of new viewers requesting the stream 

2 based on similarity measures for users sharing the buffer. 

3 While we incorporate by reference the two Herz, ef al. patents, we wish to 

4 expand slightly here on point 4. This point suggests that the contents of a buffer 

5 should be retained even after all current viewers have viewed the content /'/either 

6 there is no demand for the space it is occupying from other content requests, or if 

7 the likelihood that it will be used again soon is higher than the likelihood that any 

8 prefetched content will be used soon. In effect, the retention decision is one 

9 which takes advantage of the fact that a desirable prefetch is already in the 

10 buffer. Excepting this observation, the similarity measures and analysis are 

1 1 included by reference to the other patent and filing. 



12 The single buffer case is then optimized by the following algorithm, applied 

13 at each discrete time step in the buffer's existence: 

14 1 . If the buffer is being used by one or more viewers, examine another 

15 buffer. 

16 2. If the buffer has been used recently, estimate the probability that it will 

17 be reused in the near future (e.g., a time interval such as B/R). If it is 

18 likely, examine another buffer and mark the buffer "RETAINED". 

19 3. If the buffer is marked "EMPTY", and a non-"DOUBLED BUFFERED" 

20 buffer is being used by the maximum number of viewers, fetch the next 
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CROSS REFERENCE TO RELATED APPLICATIONS 

1 The present application claims priority from US Provisional application 

2 60/1 99,567, filed 4/25/00. 

BACKGROUND OF THE INVENTION 

3 1 . Field of the Invention: 

4 The present invention is in the field of digital communications systems, 

5 and more particularly, communications systems which transport continuous 

6 digital media such as audio and video. 

7 2. Description of the Prior Art: 

8 One of the most important forms of information in digital communications 

9 systems is continuous media, exemplified by digital audio and digital video. An 

10 example application would be transmission of a movie or live information 

11 between a service provider and a user. The traditional method of providing such 

12 a service is the broadcast network, exemplified by broadcast television or cable 

13 television. Models for future information systems use the concept of integrated 

14 services, where voice, video and structured information services such as the 

15 world-wide web (WWW) are delivered over a single logical transport service, the 

16 packet-switched Internet. 

17 To use such a system, the audio and video information must be encoded 

18 in digital form, and packetized. To reduce the use of network resources, the 
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1 information might be first encoded in digital form and then compressed. The 

2 advantage of compression is a significant reduction in data volume, while the 

3 disadvantage is the complexity of the algorithms required for encoding and 

4 decoding the compressed information. For example, directly encoding NTSC 

5 signals from a television system requires about 100 megabits/second of 

6 bandwidth, meaning that a two hour video, at 720 megabytes per minute, would 

7 require about 100 gigabytes of storage. Encoding the signals using the MPEG 2 

8 compression scheme typically reduces the bandwidth required to about 1.5 

9 megabits/second, or 12 megabytes per minute. With such compression an entire 

10 2 hour movie could be stored in about 1.5 gigabytes of storage. MPEG 2 is 

11 designed to require significantly more computation to encode than to decode, as 

12 it is presumed that service providers can afford to perform the expensive 

13 operations once, in order that the inexpensive decoding operations can be 

14 performed many times by receivers. This assumption is clearly true for a stored 

15 video, where the encoding is done once and then decoding is done whenever the 

16 video is viewed. 

17 When a system is constructed to distribute digitized continuous media to 

18 many users, there are a number of attractive opportunities for architectural 

19 techniques which can reduce system load in addition to any gains achieved by 

20 operations performed on the media such as efficient coding. In particular, a 

21 powerful technique is multicast, where the information is not sent to all possible 

22 recipients, but rather those who indicate interest, perhaps by subscription. To the 

23 degree that sharing can be achieved, e.g., the sharing of a viewing service which 
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1 plays the video into the network, significant savings can be realized. For 

2 example, rather than sending multiple copies of the same video stream, each at 

3 1 .5 megabits/second to a significant population of users, the video might be sent 

4 once via multicast to the set. 

5 Multicasts are typically represented as acyclic directed graphs called 

6 trees, where the server lies at the root of the tree, there are a set of intermediate 

7 nodes interconnected by network links which transport information to leaf nodes, 

8 at which are located users. A key feature of the intermediate nodes is their ability 

9 to replicate information to several nodes in the direction of the information flow, 

10 so that eventually the information travels from the server to all interested leaves. 

11 This basic model assumes that the users are all interested in receiving the 

12 same information at the same time, e.g., a "scheduled" time for a broadcast 

13 event, such as a sports event or a concert. When users join the multicast at 

14 some later time, they may lose the first part of the event (which they may be 

15 interested in) as it is not saved (or "buffered"). For viewing of archived material, 

16 such as replays of videos on demand, the points at which users are interested 

17 and start the viewing will vary sufficiently that multiple time-skewed copies of the 

18 digital continuous media may be being served to users at any given time. 

19 The difficulties with this situation are two-fold. First, the server is busy 

20 sending and resending multiple multicast streams of essentially the same 

21 information. Second, the advantages of multicast accrue to the degree that 

22 information is replicated - unicast, or point-to-point transmission, can be consider 
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1 a degenerate case of multicast, and in fact will be the case when there is no 

2 shared interest in a continuous media stream. Thus, to the degree in which we 

3 can aggregate demand for continuous media streams, we can optimize overall 

4 system performance. 
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SUMMARY OF THE INVENTION 

1 In accordance with the present invention, a communications method utilizes 

2 memory areas to buffer portions of the media streams. These buffer areas are 

3 shared by user applications, with the desirable consequence of reducing 

4 workload for the server system distributing media to the user (client) applications. 

5 The preferred method allows optimal balancing of buffering delays and server 

6 loads, as well as optimal choice of buffer contents for the shared memory buffers. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

1 The novel features believed characteristic of the invention are set forth in the 

2 appended claims. The invention itself however, as well as a preferred mode of use, 

3 further objects and advantages thereof, will best be understood by reference to the 

4 following detailed description of an illustrative embodiment when read in 

5 conjunction with the accompanying drawings, wherein: 

6 Figure 1 is a block diagram illustrating multicast over a network; 

7 Figure 2 illustrates shared use of a buffer; 

8 Figure 3 is a diagram illustrating use of a double buffer; and 

9 Figure 4 is a timing diagram illustrating buffer sharing. 
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DESCRIPTION OF THE PREFERRED EMBODIMENT 

1 It will be appreciated by those skilled in the art that the following 

2 description illustrates techniques that can be utilized in various types of systems. 

3 The preferred embodiment is described in the context of an internet system such 

4 as is generally known in the art. 

5 MULTICAST TRAFFIC 

6 In a store-and-forward packet-switched network, multicast is effected by 

7 multicasting the packets of data as they arrive at a node in the multicast tree. In 

8 practice, the node is a switch, router or other device which receives a multicast 

9 packet on an input port, and sends the packet on one or more output ports in 

10 accordance with the topology of the graph describing the multicast of the data to 

1 1 users. A multicast from a server to a number of users is illustrated in Figure 1 . 

12 To perform this operation, the packet must be stored in a buffer memory. 

13 In practice, the buffer memory may contain a queue of multiple packets. In the 

14 continuous media case, a queue containing a sequence of packets represents a 

15 segment of the continuous media stream. When the queue consists of large 

16 numbers of packets, substantial storage of media can be achieved within the 

17 constraints of the node's memory capacity. 

18 While it is often the case that the buffering is mainly used to effect flow 

19 control, that is, to mitigate rate differences between senders and receivers, the 

20 presence of buffering capability in nodes allows other optimizations where the 
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1 buffer capacity, e.g., the queued packets, can be shared or reused effectively. 

2 The advantage of sharing and reuse is that once the packet has been transmitted 

3 to a buffer, it need not be transmitted from the server to service a user if the user 

4 can be satisfied with the buffer contents rather than with an additional 

5 independent stream sent by the server. 

6 The difficulty in buffer reuse is independent start times for the multicast 

7 streams, as what is in the queue of packets destined to a particular user may not 

8 be the same data as that required by another user. To the degree that work (e.g., 

9 the transmission of data from one node to another) can be avoided through 

10 sharing, the system will perform better. An example of better performance would 

11 be more concurrent users operating with a reduction in bandwidth relative to 

12 similar systems. 

13 SHARED BUFFERS 

14 Each segment of buffered continuous media represents the result of work done 

15 by the network to transport the digitized continuous media to the buffer. Thus, to 

16 the degree that the contents of the buffer can be shared, the work of transporting 

17 the continuous media to the buffer can be shared. An essential observation is 

18 that there is a clear relationship between buffer occupancy and time: each buffer 

19 segment of size B bytes represents a playout time of B/R seconds, where R is 

20 the encoding rate in bytes per second. A related observation was made in John 

21 H. Shaffer, "The Effects of High Speed Netwoks on Wide Area Distributed 

22 Systems", Ph.D. Thesis, CIS, University of Pennsylvania, 1996. 
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1 The central observation used in our approach is that if additional viewers 

2 arrive within B/R seconds of the start time of the original playout, these additional 

3 viewers can utilize the contents of the buffer segment rather than requiring 

4 additional network bandwidth for transmission. To be specific, if the arrival time of 

5 viewers V1 and V2 is separated by less than B/R, they can share a buffer 

6 segment. This is illustrated in Figure 2. 

7 While a related idea was employed in U. Legezda, D. Wetherall and J. 

8 Guttag, "Active Reliable Multicast, Proc. Infocomm 1998, SF, CA to reduce 

9 required bandwidth in a reliable multicast, the sharing was different, in that it was 

10 used for recovery of lost multicast packets rather than as a support mechanism 

1 1 for multicast of digital continuous media. The key basic point of our method is the 

12 use of a per-recipient pointer (an index) into the shared buffer to reflect the 

13 differences in start times. By use of this pointer, the buffer contents can be 

14 effectively shared. 

15 There are a variety of parameters which can be adjusted in the design of 

16 such a system. First, to the degree that whole instances of digital continuous 

17 media (for convenience, we will call these "files") can be buffered, there is 

18 significant advantage to be had in that the buffer management algorithms are 

19 simplified. This is because the algorithms need to spend less effort managing the 

20 differences in buffer refresh rate caused by supporting multiple start times. This 

21 management cost is incurred if the whole file is not available, as to save 

22 bandwidth, the buffer contents must be retained until the last viewer is done with 

23 them, i.e., their pointer has advanced to the end of the buffer. This problem is 
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1 easily addressed if the well-known technique of double-buffering is employed to 

2 manage two buffers of size B and the start time deltas are limited to B/R, as 

3 above. In the double-buffering technique, one buffer is drained by the viewers 

4 while the other is filled from the network, and this solution allows the time- 

5 separated viewers to share the 2*B space. The technique is illustrated in Figure 

6 3. 



7 The same effect can be achieved by limiting the time differences between 

8 V1 and V2 to B/(2*R). 

9 OPTIMAL SHARING OF A SINGLE BUFFER 

10 There are a variety of techniques for buffer management which can make 



11 use of the buffer capacity to better store continuous digital media. To review 

12 some of the techniques used in US Patent 5,754,938, issued May 19, 1998 to 

13 Herz et al, and US Application No. 09/024,278, filed Feb 17, 1998, both hereby 

14 incorporated by reference hereinto, the buffer contents can be: 



15 1. Selected based on statistical modeling of the user based on similarity 

16 measures derived from previous viewing. 

17 2. Can be prefetched in advance of viewing demand in order to smooth 

18 demands on bandwidth. 

19 3. Can be prefetched in anticipation if possible viewing demand based on 

20 similarity measures for the viewers sharing the buffer. 
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1 4. Can be retained in anticipation of new viewers requesting the stream 

2 based on similarity measures for users sharing the buffer. 

3 While we incorporate by reference the two Herz, et al. patents, we wish to 

4 expand slightly here on point 4. This point suggests that the contents of a buffer 

5 should be retained even after all current viewers have viewed the content //either 

6 there is no demand for the space it is occupying from other content requests, or if 

7 the likelihood that it will be used again soon is higher than the likelihood that any 

8 prefetched content will be used soon. In effect, the retention decision is one 

9 which takes advantage of the fact that a desirable prefetch is already in the 

10 buffer. Excepting this observation, the similarity measures and analysis are 

1 1 included by reference to the other patent and filing. 



12 The single buffer case is then optimized by the following algorithm, applied 

13 at each discrete time step in the buffer's existence: 

14 1 . if the buffer is being used by one or more viewers, examine another 

15 buffer. 

16 2. If the buffer has been used recently, estimate the probability that it will 

17 be reused in the near future (e.g., a time interval such as B/R). If it is 

18 likely, examine another buffer and mark the buffer "RETAINED". 

19 3. If the buffer is marked "EMPTY", and a non-"DOUBLED BUFFERED" 

20 buffer is being used by the maximum number of viewers, fetch the next 
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1 



1 B bytes of the continuous media stream into the new buffer to achieve 

2 double buffering and mark both buffers "DOUBLE BUFFERED". 

3 4. If the buffer is marked "EMPTY" and similarity measures show that 

4 prefetching more of the stream would optimize performance of the 

5 users of the buffer, mark the buffer "PREFETCHING" and request that 

6 a continuous media stream of B/R duration be sent to fill the buffer. 

7 5. If a buffer is PREFETCHING and a buffer is required for on-demand 

8 traffic, mark the buffer "EMPTY". 

9 6. If the buffer has been used, and it is unlikely to be reused in the near 

10 future, mark it "EMPTY". 

11 7. If the buffer has not been used, mark it "EMPTY". 

12 An interesting consequence of this algorithm is that highly popular data 



13 will be either completely prefetched or after prefetching and/or viewing, wholly 

14 retained. Thus, without explicitly requiring whole file caching, the system will 

15 naturally achieve it, and will do so on the basis of statistical usage and estimation 

16 measures. 

17 There are additional gains to be made because streams are generally 

18 served over relatively lengthy periods of time and are consumed sequentially. 

19 This is especially true for such data types as audio and video programming. 

20 Because the full stream is delivered to a particular point, it is possible to predict 

21 content demand quite precisely for the following minutes or even hours. 
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1 Moreover, when geographically proximate nodes are conveying the same 

2 stream, even at a time shift, it is possible for them to economize on upstream 

3 bandwidth. 

4 For example, if node N1 in the network is relaying frames from the FIRST 

5 hour of Citizen Kane to one or more downstream users, then the probability is 

6 extremely high that it will eventually have to relay frames from the SECOND hour 

7 in Citizen Kane, and we can accurately estimate when it will have to do so. 

8 If some nearby node N2 happens to have such frames in a buffer (e.g., N2 

9 is already relaying the second hour), then it is cheap for N1 to get those frames. 

10 This cheapness is temporary: N2 isn't going to keep the frames forever, so it may 

1 1 make sense for N1 to immediately copy N2's frames - that is, to prefetch them. 

12 In other words, N1 is getting its stream from the head end, but it is also 

13 monitoring N2's stream and caching it for later. Eventually N1 will be able to 

14 switch to the cache, at which point it can drop the head end. 

15 Note that the streams don't need to be transmitted at the same speed at 

16 which the user will consume them - with additional bandwidth, it would be 

17 possible, for example, to send a node the remaining head end of a video stream 

18 in a rapid burst. If multiple streams are being fed to a localized cluster of nodes, 

19 such a bursting strategy would allow the many streams to be rapidly collapsed 

20 into very few (if not one) shared streams, greatly decreasing bandwidth use by 

21 the locale. 
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1 More complex probabilistic strategies are also possible, with each node 

2 participating in a self-organized market, requesting and taking bids for slices of 

3 bandwidth. 

4 OPTIMAL SYSTEM DESIGN WITH BUFFERS AND SERVERS 

5 It is obvious that the scheme can be applied to a server with a single 

6 buffer. When a more complex system, such as that in Figure 1 is constructed, 

7 the buffers can be viewed as a shared resource and as multiple layers of 

8 intermediate capability. In such a system, we can use a hierarchical scheme, so 

9 that individual viewers using a single buffer are replaced by multiple viewers 

10 sharing a buffer, which in turn shares buffers in the multicast tree. Further, the 

11 buffers can cooperate amongst themselves to share caching capacity, and 

12 further, the similarity measures of the patents included by reference can be 

13 localized to the population of viewers near the particular buffer. In such a case, 

14 the population statistics are localized. Caches that are higher in the tree (nearer 

15 to the server) aggregate more and more traffic, but have better models of user 

16 demand from the aggregated demand of the buffers which are presenting 

17 aggregated demand to them. 

18 Additionally, multiple servers can be used. In this case, buffers may 

19 communicate with multiple servers based on the demands of their users, and 

20 prefetch information based both on user interest and aggregated interest, and 

21 capacity of the server being used. 
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1 The advantage of these architectural advances for scaling is significant. 

2 Information in caches near the edges of the system is localized to the user 

3 interests represented by the similarity measures applied to the continuous digital 

4 media or descriptive information associated with it. Each level of buffering in the 

5 hierarchical multicast demand aggregates various levels of interest and optimism 

6 in prefetching data: data prefetched by many buffer caches on behalf of their 

7 clients/viewers will be more likely to be buffer resident where higher levels of 

8 aggregation ensue. 

9 VI Demand Aggregation of Data Streams as an Optimally Bandwidth Conserving 

10 Form of Pre-Fetching 

11 The following technical methodology describing similarity-informed pre- 

12 fetching (the subject of a co-pending patent application by the inventors of the 

13 presently disclosed invention) provides an underlying technical framework for the 

14 object of the novel innovative concept introduced within this section which is the 

15 aggregation of data streams based upon capitalizing upon aggregate demand 

16 prediction of a sub-population of users who are imminently likely to request a 

17 particular file (a standard file or a streaming file). Similarity-informed pre-fetching 

18 provides a fundamental technical basis for demand aggregation of down-loaded 

19 data streams with a few fundamental differences as are explained further below. 

20 Bandwidth is greater at the root end of a hierarchical node network 

21 system, in anticipation of a request, it is prudent to use the similarity measure to 

22 predictively cache files into local servers and further to narrow-cast selections 
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1 into given distribution cells and (hierarchical) sub-cells through sub-servers 

2 based on what selections are most likely to be requested in each cell so as to 

3 significantly increase the utilization of bandwidth via this hierarchical narrow-cast 

4 configuration. The importance of this savings appears in proportion to the degree 

5 of granularity (smallness of the cell) in the narrow-cast architecture. This 

6 technique can also be used to make decisions for scheduling what data should 

7 be placed on dedicated channels. This may be more network-efficient if a file 

8 were popular enough to be continuously in the queue because upon submission 

9 of a request a file may be partially downloaded regardless of where in the length 

10 of the file the download began. The (initially) missed portion of the file can then 

11 be immediately picked up as the file narrow-cast starts over, thus completing the 

12 download in the same time period as if a special request for the file were made. 

13 Mobile users whose geographic locations are known can have files pre-cached 

14 (e.g. at night) into the servers which are presently physically closest to them at 

15 any given time. 

16 The present data distribution system employs the idea of pre-fetching, 

17 which has also been referred to as pre-caching, cache-pre-loading, or 

18 anticipation in the technical literature. The basic idea is that if good predictions of 

19 future data requirements are available, and there is excess data-fetching 

20 capability available, the data should be fetched aggressively in anticipation of 

21 future needs. If successful, this technique has two major benefits applicable to 

22 present and future networks. First, it can reduce (i.e., improve) response-time, a 

23 major performance advantage in interactive systems. Second, it can reduce 
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1 congestion and other problems associated with network overload. To 

2 understand how the responsiveness of the system is improved, the unused 

3 bandwidth can be used to transmit information likely to be used in the future. For 

4 example, if a list is being traversed 1,2,3,4, it is likely that if object N has been 

5 requested, that object N+1 will be the next request. If N+1 is pre-fetched from 

6 the remote system, it will be available when the request is made with additional 

7 delays. All of the "UNUSED BANDWIDTH" can potentially be used to pre-fetch. 

8 Within the context of the pending patent on the pre-fetching concept 

9 entitled "Broadcast Data Distribution System with Asymmetric Uplink/Downlink 

10 Bandwidths" one of the key objectives is it involves the use of pre-fetching as a 

11 congestion control technique as if we pre-fetch successfully during more lightly 

12 loaded periods (such as TIME=0.42), we reduce the probability of data being 

13 requested in the futures, essentially trading the guarantee of a fully loaded 

14 network today for the promise of no congestion in the future. By fetching data in 

15 anticipation of future needs, we reduce (at least probabilistically) those future 

16 needs. 

17 Pre-fetching has been used in the computer operating systems field for 

18 several decades, and a variety of algorithms have been explored. A. J. Smith of 

19 Berkeley has reported that the only case where successful predictions about 

20 future requests for memory objects can be made is when accesses are 

21 sequential. More recent work for higher-level content such as World-Wide Web 

22 (WWW) hypertext has shown that user-authored links to other hypermedia 

23 documents can be followed with some success. 
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1 The pre-fetching technology which the inventors of this disclosure have 

2 previously invented is based on unused slots being filled with pre-sent 

3 information based on our understanding of user interest using the similarity 

4 measures developed in the issued patent entitled "System for Generation of User 

5 Profiles for a System for Customized Electronic Identification of Desirable 

6 Objects" (US Patent 5,754,939)and used for our prioritization (see p. 18 of the 

7 above-referenced invention disclosure), a concept that they do not deal with, as 

8 they follow http: links based on observations about the high probability that these 

9 links will be followed by users. 

10 The pre-fetching invention may be usefully applied within the context of 

11 set-top box like devices such as personal digital assistants or network 

12 computers, or personal computers used as a form of set-top box as well as any 

13 type of "fat client" as a method of reducing response time as observed by users. 

14 This method used "links" to other documents embedded in an HTTP-format file 

15 as hints that those links should be followed in pre-fetching data; that is, the linked 

16 documents should be fetched in anticipation of the user's desire to follow the 

17 links to those documents. 

18 The present invention provides two enhancements to this scheme. First, it 

19 provides a technological means by which the pre-fetched data can be intermixed 

20 with on-demand data to provide overall improvements in response time to a large 

21 population of HTTP/WWW users, with reduced memory requirements. Second, 

22 t-he present invention, which views the down-link as a fixed capacity resource, 

23 provides a general scheduling method embodying techniques such as user 
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1 preferences to pre-fetch when slots or bandwidth are underutilized, to 

2 preemptively reduce future demand for bandwidth. 



3 In addition, the similarity measures which suggest: 

4 1 . An anticipated behavioral similarity between different though metrically 

5 "similar" users may be used to further analyze other previous user's 

6 on-line behavior as well as, 

7 2. Page similarity to other links, which the user has an explicitly or 

8 implicitly (predicted) affinity towards, may be a technique to further 

9 improve predictive accuracy as to which of the available links the user 

10 is probabilistically most likely to imminently select next (compare to 

11 aggregatively using the overall popularity of those links). 

12 The general technique of using similarity-informed pre-fetching is 

13 described at length in US patent entitled "Pseudonymous Server For System For 

14 Customized Electronic Identification Of Desirable Objects", U.S. Patent No. 

15 5,754,938 filed October 31 , 1 995, issued May 1 9, 1 998. 

16 The technique of data-streams is based upon a similar variation of the 

17 type of similarity-informed pre-fetching, which is performed on a relatively 

18 dynamic basis as suggested below. A couple of fundamental extensions to the 

19 basic technique are added, however. Unlike dynamic similarity-based pre- 

20 fetching, because there is a certain degree of predictive error, which invariably 

21 occurs though the present techniques attempt to anticipate imminently 
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1 forthcoming file requests on the part of the user (and where the error rate 

2 exponentially increases in proportion to the length of the anticipatory period), the 

3 basic goal of this dynamic pre-fetching is increasing speed of user access to 

4 page requests at the expense of the additional bandwidth, which is consumed as 

5 a result. However, in the case of data stream aggregation the key objective is to 

6 minimize bandwidth utilization. This approach also results in the ability to not 

7 adversely affect speed of access to the data. Although its use is in no way 

8 limited, it is likely that the present approach to pre-fetching basic data stream 

9 aggregation may be particularly well suited for delivering data to the leaf end 

10 nodes of the network 

11 Compared to similarity-based pre-fetching, the key modification in the 

12 basic method (and certainly technical challenge) of pre-fetching based data 

13 stream aggregation is the following: 

14 In similarity-based dynamic pre-fetching (as suggested above), the 

15 similarity measurements of the predictive data model anticipates, on a dynamic 

16 basis, the forthcoming file request actions of the user. In fact the outputs, i.e., 

17 probabilities of a given user requesting any given file, can be measured as a 

18 function of time (T). If this measurement is based upon the entire subset of that 

19 user population which has at least some non-zero probability of imminently 

20 selecting that file, we, in turn, may determine the aggregate probability of that file 

21 to be requested by the entire user sub-population (i.e., which is serviced by a 

22 particular data distribution server) as a function of time. Of course, in 

23 accordance with the probabilistic model, this aggregate probability curve (of 
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1 average likelihood of the user population to initiate the request) changes on a 

2 moment to moment basis changes in accordance with further behavioral 

3 feedback of the sub-population as time (t) approaches the average 

4 (probabilistically most likely) time for overall demand for that file to culminate 

5 (however, this value remains fixed for our purposes as once a pre-fetching 

6 decision has been made any subsequent probability shifts are irrelevant). The 

7 point in time at which a data stream is scheduled to initiate is Ti and the end is 

8 T 2 . There is another important relationship which is within a given average 

9 probability measure the actual time from the point in time which is that moment 

10 that the user population on average is anticipated to actually request the file. We 

11 will call this value T M . The end of that period is called T b2 . We want to also 

12 determine the average effective available memory of each relevant user's client's 

13 local buffer, however, this value is also affected by such variables which compete 

14 for this space such as how much of this memory had been pre-allowed to long- 

15 term pre-fetching (the relative proportions of such allocation of which would be 

16 determined through experimentation and may be network specific) and how 

17 much "risk diversification" is provided for i.e., the probability distribution for any 

18 given individual at any give instant in time (t) preceeding an actual request is very 

19 likely to contain secondarily another (or other) file(s) with some non-zero 

20 probability. Thus the total probability (and possibly relative probability) 

21 determines available buffer memory for the present anticipated file request. For 

22 purposes of the present estimation, this value is called AT b and is measured as 

23 the amount of time that the present effective available memory buffer for that 
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1 client is able to receive the data stream associated with the present anticipated 

2 forthcoming request. 

3 We then want to select a time T to actually request the file for delivery to 

4 all the relevant users U based upon the average of the product of probability (of 

5 making the desired relevant file request) (P b ) and time such that TYs value is 

6 such that period. 

7 Ti - T 2 provides the maximum possible product of probability and time for 

8 all relevant clients' buffers collectively (that are able to concurrently co-exist 

9 within period Ti - T 2 ). (the fixed value describing the period of the data stream) 

10 Based upon these variables the key technical challenge of pre-fetching-based 

11 data stream aggregation is determining values of Ti - T 2 which achieve 

12 optimality in reducing bandwidth consumption in the multi-cast of that particular 

13 data stream. This is represented in Fig. 4 by our attempt to find a T^ value 

14 optimizing the area described by probability and time within the T^ - T 2 period. 

15 The following equation is provided: 

16 (Ti - T 2 ) = {AT b P b Z + (AT b + 1 ) (P b +1 )(Z + 1 ) +. ..}(mean average) 

17 where Z is the percentage of T b i - T b2 which overlaps with Tt - T 2 . 

18 Fig. 4 depicts graphically how a key objective of the above equation is to 

19 find a Ti - T 2 value which maximizes the (2 D) area under the various client 

20 buffers collectively in a probability time graph. 
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1 Finally, in light of the present technique because the predictive models are 

2 prone to a certain degree of oversight, i.e., not anticipating the forthcoming 

3 request actions certain uses for a given file or not the accurate timing therefore 

4 (e.g., not anticipating the request action soon enough) there will invariably exist 

5 certain inefficiencies in the present model in which certain user request data 

6 streams are not properly (or not at all) aggregated, thus, we would like to provide 

7 yet another additional solution to more efficiently aggregate these inefficiently 

8 transmitted streams. The idea is that if a local client buffer has failed to initiate 

9 download of a stream (or has initiated it after its initialization, we can effectively 

10 re-transmit the "missed" portion of the stream to the new requester at a very fast 

11 rate (e.g., if it is streaming media content, considerably faster than real-time) up 

12 until the point in which the data received by the new requester "catches up" with 

13 the original stream at which time both the new requester and original requester(s) 

14 then share the same stream for the remainder of its duration. In such an event 

15 the bandwidth utilization specifically allocated to that missed portion of the file we 

16 could say becomes "bursty". Due to the somewhat different mechanism of both 

17 data stream aggregation methods, in trying to achieve optimal bandwidth 

18 utilization the degree to which this additional approach to aggregation should be 

19 relied upon compared to the original pre-fetching based approach (that is to say 

20 from a probabilistic standpoint how speculatively do we want to pre-fetch versus 

21 rely upon effectively the present ("fall-back") approach of using burstiness of 

22 transmission to compensate for the resulting "missed" requests of the predictive 

23 (anticipatory) approach. This balance between the degree of utilization of these 
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1 two methods my also be a bit subjective in as much as if bandwidth is (presently) 

2 overly constrained, the system may automatically adjust by relying more heavily 

3 upon the original technique of pre-fetching based data stream aggregation (and 

4 certainly even more towards artificial delays in as much as bursty data stream 

5 aggregation while involving no delays in initiating does involve some degree of 

6 extra bandwidth utilization during the "bursts". Of course, as part of this pre- 

7 fetching procedure and (relatedly) the ) the number of files provided a given 

8 probability distribution for forthcoming requests of those files must also be 

9 determined through further experimentation. 

10 In another form of data stream aggregation called "artificial delays", 

11 described further below, speed of access to the user is invariably compromised 

12 in direct proportion to the degree of data stream aggregation which is desired for 

13 reducing bandwidth. In the presently described version of data stream 

14 aggregation (as is also true in the case of artificial delays), although it may be 

15 variable, on the average, the number of users serviced by the data distribution 

16 server as well as the degree of popularity of the particular file being presently 

17 requested by the user relative to that particular population of users, is directly 

18 proportional to the speed of access by the user to that particular requested file. 

19 Accordingly, non-dynamic pre-fetching (also detailed above), because it is both 

20 non-dynamic in nature and also achieves a reduction in overall network traffic 

21 can be used in combination with the present pre-fetching based data stream 

22 aggregation in order to provide an optimally efficient traffic reduced network 

23 environment, and as is further described below there are further bandwidth 
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1 reduction optimization techniques which allow for the approach between the 

2 intermediate node and leaf end of the network while allowing for non-dynamic 

3 similarity-informed pre-caching and the most bandwidth efficient data stream 

4 aggregation technique, i. e., artificial delays to occur on nodes close to "trunk" 

5 portion of the network where the potential number of user connections 

6 represented is extremely large and thus potential for bandwidth conservation, 

7 using artificial delays is extremely great. On the other hand, if near the leaf end 

8 of the network further bandwidth conservation is desired, it is even possible to 

9 provide the technique of artificial delays in combination with static pre-caching 

10 and dynamic pre-fetching based data stream aggregation. And this approach 

11 may be particularly compelling in achieving substantial bandwidth conservation if 

12 the end-user population (or number of network "leaves") is quite large. 

13 Artificial Delays 

14 This note discusses the notion of "artificial delays" in the queuing of 

15 requests for the satellite or cable system to which our set top boxes are attached. 

16 The idea is that by careful management of the queues, we can effect significant 

17 bandwidth savings for the system as a whole. If you will remember, the Server 

18 scheduling algorithm (I've attached the text for it from the DBS scheme I sent 

19 during the Summer at the end of this e-mail) goes like this: 

20 The client set-top box (of which there should be many) sends REQUESTS 

21 for information in cell-sized units to the server system. The server system applies 

22 a priority algorithm (see especially Step 10, below) to CHOOSE the next cell to 
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1 send. By design of the relative priorities, we can get good responsiveness and 

2 reduced bandwidth needs, in spite of the low memory needs (and low cost) of the 

3 set top boxes. 

4 Imagine the scenario where there are MANY set tops connected to the 

5 server. This situation might be where the Cs are Clients and the S is a Server. 

6 Now clearly, there is a multiplicity of Clients, and by virtue of this multiplicity, we 

7 may be able to achieve a savings through appropriate delays. I again believe the 

8 similarity measure is the key to success here, and to novelty. Consider the cell 

9 requests for Clients C1, C4 and C5, shown below using letters to indicate 

10 particular cells as discussed in our disclosure text. 

11 C1: E-T-A-O-l-N-S-H-R-D-L-U 

12 C4: N-A-T-l-O-N-A-L-V-E-L 

13 C5: E-A-T-O-N-L-Y-S-U-D 

14 We mark these cell request with times associated with their transmission 

15 intervals: 

16 T: 

17 1:000000000 1 1 1 1 

18 M:1 -2-3-4-5-6-7-8-9-0-1 -2-3 

19 E: 

20 Now, for convenience, assume that all of the cell requests show above 

21 have the same priority. Then the server might actually send the following 

22 sequence of cells over the channel: 
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1 S:E-N-E-T-A-A-A-T-T-0-l-0 

2 Thus, we are servicing the cell requests C1-C4-C5-C1-C4-C5 (in fact, 

3 the Server may notice the overlaps between requests by C1 and C5 in the first 

4 interval, C4 and C5 in the second interval, C4 and C5 in the third interval, and C1 

5 and C5 in the fourth interval, giving: 

6 S:E-N-T-A-A-T-0-l-l-0-N-N... 

7 ) 

8 Imagine that the clients are always listening. Then, we can delay cell 

9 requests in the HOPE that the REPLYS can be MERGED, satisfying multiple set- 

10 top-box clients with the same REPLY. To make this concrete, consider delaying 

11 service by one period. So the output of the server then looks like: 

12 S:#-E-N-T-A-0-l-N-L-S-A-Y.... 

13 What is going on here is very subtle. By delaying some clients service 



14 requests, we are INCREASING THE PROBABILITY that another such request 

15 will come in, which can we can fold into service of the equivalent delayed 

16 request. The cost is potentially in delay, but with enough overlap, the cell times 

17 are short enough for 48 byes on a DBS channel that we can probably delay 

18 significantly. 

19 Considering the problem theoretically for a moment, we can compute the 

20 gain for the an acceptable delay D as being the number of redundant 

21 transmissions which are eliminated due to a delay D. So, for delays of from 1 to 
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1 10 cell times the total number of DBS cells without redundancy checks is 30; the 

2 number required when this small optimization is applied is as shown below: 



3 Delay DBS CellsBandwidth Savings 

4 

5 02430/24=25% 

6 11830/18=66% 

7 21730/17=76% 

8 31730/17=76% 

9 41530/15=100% 

10 51530/15=100% 

11 61530/15=100% 

12 71530/15=100% 

13 81530/15=100% 

14 91530/15=100% 

15 101430/14=114% 

16 We compute the bandwidth gain against the dumb use of 30 cell times; 



17 the bandwidth gain comes from the fact that the synchronous satellite channel 

18 gives us a fixed bandwidth, giving a fixed number of cells per unit time, and we 

19 have just saved 16 cell times by use of the delay scheme. For this example, at 

20 this point no more gain is possible, since all the duplication has been eliminated. 

21 In some sense, this behaves like a compression scheme. The similarity 

22 algorithm increases the probability that these overlaps will occur - the ideal 

23 situation is we are waiting long enough so that the scheduled broadcast cell 
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1 satisfies almost all requests for that cell within a significant time intervals (say 

2 several milliseconds). 

3 Optimal Stream-Handling Techniques as a Function of Node Location in the 

4 Network. 

5 The method by which streams are handled depends in part on a node's 

6 location in the network. 

7 A node close to the network's "trunk" will often be best served by artificial 

8 delays and pre-caching techniques, whereas nodes closer to the network's 

9 "leaves" more efficiently make use of demand aggregation and pre-fetching. 

10 The particular topology of a given network will determine the distance a 

11 node must be from the center in order for the "leaf" approaches to be more 

12 appropriate than the "trunk" approaches. 

13 Long-Term vs. Short-Term Pre-Caching - 

14 Due to the relative importance of responsiveness compared to relative 

15 bandwidth savings opportunities at the edges (leaf ends) of the network compared 

16 to the trunk , dynamic pre-fetching (and pre-fetching-based data stream 

17 aggregation) may be efficiently utilized near the network edges while more static 

18 (long-term) pre-caching )may be more appropriate closer to the trunk nodes of the 

19 network. Also at this trunk level because the number of repetitive requests artificial 

20 delay based data aggregation is very efficient and because of the rather large 
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1 storage capacity associated with this repetitive file traffic, long-term pre-caching is 

2 certainly ideal. 

3 Automatic Adaptive Shifting to Other Techniques - 

4 E. g., if/when traffic congestion and slow down at the edges occurs, it may 

5 be prudent to shift the relative degree of utilization of one technique to another , 

6 e.g., in the present scenario the use of artificial delays may be able to 

7 significantly reduce delays through relieving congestion. Pre-fetching-based data 

8 stream aggregation may also provide useful advantages. 

9 As a result, different levels in the network have different traffic and storage 

10 characteristics (and relative responsiveness priorities). The selection and relative 

11 dependence upon one of the above techniques versus another may be different 

12 at these different levels and the desirable optimizations requiring different uses 

13 and priorities of these various techniques may change at any given level in the 

14 network and at any given time based upon these dynamically changing 

15 characteristics of the network traffic (to the extent that these dynamic 

16 characteristic changes are not pre-determinable using existing network traffic 

17 intelligence solutions). 

18 Likewise, because both probability and statistical confidence (degree of 

19 certainty) varies even between different files which are predictively anticipated 

20 (using pre-fetching-based data stream aggregation), the use of this method (the 

21 degree of reliance) of this method, may vary even between anticipated file 

22 requestion, e.g., it may be advantageous if the probability of a file being requested 
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1 is low or the statistical confidence of a file (even the most "likely" file) to make the 

2 anticipatory period for requesting that file low (i.e., less speculative) in order to rely 

3 more upon bursty data stream aggregation (or artificial delays particularly in these 

4 latter two techniques the period of anticipation may be ideally further minimized 

5 if/when these other techniques are utilized with higher relative importance. 

6 Consideration for Pre-Fetching Based Data Stream Aggregation - 

7 In addition, because data stream pre-fetching aggregation is a very important 

8 technique for providing bandwidth conservation (an important need for the leaf 

9 ends of the network), while also providing a high degree of responsiveness to 

10 requests, it may also be important to insure that the appropriate technique (or 

11 combination of techniques) as suggested above is utilized to insure that files 

12 reach this bandwidth bottleneck in the network, (e.g., the intermediate node just 

13 prior to leaf end node) such that pre-fetching data stream aggregation at this 

14 level can occur extremely efficiently without uneccessary efficiency 

15 consequences resulting from delays higher up in the network (e. g., this may be 

16 especially true in the case of the wireless network example). 

17 ADDITIONAL CONSIDERATIONS 

18 1. Streaming vs. Static Files 

19 This description has been focused mainly on streaming types of data. 

20 However, packet analysis at the backbone level could identify metrically close 

21 data transmissions of any sort (that is, streaming or static ("static" meaning non- 
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1 streaming). Thus, when two proximate nodes download the same extremely 

2 large data file of a non-streaming sort, they could use a shared buffer to reduce 

3 their external need of bandwidth to a single connection. In the end, this 

4 architecture handles files of both the streaming and static types in very much the 

5 same way. 

6 2. Tradeoffs between Bandwidth and Memory 

7 In all of these examples, there is a continuous tradeoff between bandwidth 

8 and memory: if a great deal of bandwidth is available, there is little need for 

9 localized buffers - every user can afford an individual real-time connection and 

10 therefore has no need to store any streams. On the other hand, if a great deal of 

1 1 memory is available, the permanent storage of all incoming streams to a local 

12 server would eventually reduce the need for external bandwidth. 

13 EXEMPLARY APPLICATIONS 

14 The network architecture described in this patent can be embodied in 

15 many relevant applications. Note that although a few are described here, the 

16 architecture is generally applicable to any situation in which multiple proximate 

17 (relative to the network) users will potentially access the same stream of data 

18 within some period of time. 

19 1. Neighborhood Server 

20 A subdivision of homes is linked by high-bandwidth lines to a shared, local 

21 server which acts as the neighborhood's central portal to the Internet. Using 
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1 residents' video preference profiles, the server predicts which movies or 

2 television programs are most likely to be requested as downloads, and buffers 

3 (for example) the first 1 / 2 hour of the 50 most likely selections to local memory. 

4 This could best be done during the day, when residents are more likely to be at 

5 work, temporally smoothing the neighborhood's demand for bandwidth. 

6 In the evening, if a resident happens to select a video stream which has 

7 already been front-loaded in the local server, he is sent data directly from the 

8 local buffer. If no other resident requests the same program, the single user is 

9 switched to an external stream when the buffer has been exhausted. However, if 

10 several residents select the same program at roughly the same time (that is, 

11 within the 1 /s hour buffer), the server continuously downloads (and buffers) a 

12 window of streamed data that spans the users' timings. For example, Resident A 

13 starts watching Citizen Kane; for the first Vz hour he is initially fed a stream from 

14 the local server. Suppose 5 minutes later Resident B also starts to watch Citizen 

15 Kane. When A reaches the end of the Vz hour buffer, the neighborhood server 

16 starts to draw Citizen Kane as a stream directly from the internet, pushing it to A 

17 directly, and then saving the stream to a continuously-refreshed five-minute 

18 buffer. B is fed from the end of the buffer, just before the stream is finally cleared 

19 from memory. In this way, rather than opening two high-bandwidth connections 

20 to the Internet, the local neighborhood server needs only open a single high- 

21 bandwidth connection and allocate enough memory in its local buffer to hold five 

22 minutes of video programming. When multiple residents watch the same 
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1 programming at fairly similar times, this method greatly reduces the subdivision's 

2 overall need for external bandwidth. 

3 Note too that peer-to-peer (P2P) methods could be used to expand the 

4 neighborhood's available storage - the neighborhood server would be given 

5 permission by residents to temporarily make use of memory or hard disk space 

6 that they are not currently using on their own home machines. This would 

7 expand the number of stream front-ends that could be prefetched. 

8 2. Demand Aggregation for Wireless Electronics 

9 Demand aggregation techniques would also be useful in the case of 

10 wireless devices -- if many users in a particular locale exhibit similar data needs 

11 (such as Wall Street executives checking popular stock prices periodically), 

12 bandwidth could be conserved by continuously sending the information in a 

13 single stream to a server connected to the local wireless transmitter. 

14 3, Optimization of Web Page Delivery 

15 While a user peruses a given Web page, it would be possible to prefetch 

16 many of the pages to which his current page is currently linked. Then, when the 

17 user clicks a hyperlink, because his selection is already in the local buffer it can 

18 be delivered almost instantly. Obviously, this could be made more sophisticated, 

19 with probabilistic methods used to determine which pages are most likely to next 

20 be chosen by the user, and thus which are the most logical candidates for 

21 prefetching. 
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While the invention has been particularly shown and described with 
reference to a preferred embodiment, it will be understood by those skilled in the art 
that various changes in form and detail may be made therein without departing from 
the spirit and scope of the invention. 
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